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Abstract 

This  work  describes  scalable  coupling  of  two  stand 
alone  computer  codes  for  multi-scale  impact  analysis  of 
composites.  An  asymptotic  expansion  homogenization 
(AEH)  based  micro-structural  code  available  for 
modeling  micro-structural  aspects  of  modern  armor 
materials  is  coupled  with  PARADYN,  a parallel  explicit 
Lagrangian  finite  element  code.  The  first  code  enables 
modeling  of  material  micro-structures  in  simple  loading 
situations  in  stand  alone  form.  The  coupling  of  this  code 
to  PARADYN,  which  is  a parallel  version  of  the  Lawrence 
Livermore  National  Laboratory  (LLNL)  's  serial 
DYNA3D,  enables  a micro-macro  type  multi-scale 
analysis  of  large  elastic-plastic  deformation  response  of 
structures  under  generalized  three  dimensional  impact 
conditions.  Three  sets  of  results  are  presented  to 
demonstrate:  1)  the  verification  of  the  AEH-PARADYN 
model  coupling  to  PARADYN,  2)  validation  of  the  AEH, 
and  3)  the  scalability  of  the  coupled  model. 

1.  Introduction 

Spanning  a composite  from  its  micro-phases’  level  to 
its  global  structural  level  with  finite  elements  lead  to  large 
meshes;  the  resulting  computations  are  both  expensive 
and  intractable.  Multi-scale  analyses  are  used  to  alleviate 
this  problem.  At  a basic  level,  these  analyses  express  the 
micro-level  response  as  a local  perturbation  of  global 
response.  While  many  theories  are  available  for  linking 
the  micro-structural  response  to  the  global  response,  the 
AEH  theory  used  in  the  first  code  is  extended  by  Chung, 
Namburu,  and  Tamma,  (1999)  to  armor  composite 
impacts. 

In  the  AEH  approach,  the  micro-level  response  was 
expressed  in  terms  of  the  global  response  in  a strict, 
mathematically  seamless  approach.  An  updated 
Lagrangian  scheme  is  employed  to  account  for  large 
displacements,  strains,  and  rotations.  At  each  time  step. 


the  equilibrium  solution  from  the  previous  time  step  is 
used  to  update  the  local  micro-structure.  Since  explicit 
time  integrations  take  quite  a large  number  of  time  steps 
and  micro-level  computations  are  to  be  performed  at  each 
and  every  time  step,  scalable  computational  approaches 
are  needed  to  use  this  computationally  intensive  method. 
With  the  aid  of  domain  decomposition  and  Message 
Passing  Interface  (MPI),  the  basic  equations  of  Chung 
(1999)  were  re-cast  for  parallel  computation  and  were 
shown  to  be  scalable  by  Valisetty  (2000)  in  a stand  alone 
code.  In  the  present  work,  this  code  is  coupled  to 
PARADYN  within  the  message  passing  interface  (MPI) 
scheme  of  the  latter.  Three  sets  of  results  are  presented  to 
demonstrate  the  verification,  validation,  and  scalability  of 
the  AEH  model  coupling  to  PARADYN. 

2.  Coupling  the  AEH  Model  in  PARADYN 

PARADYN  is  a general  purpose  explicit  dynamics 
code  with  a capability  for  defining  multiple  element  types 
and  material  types.  Each  element  of  the  global  finite 
element  model  that  is  tabbed  for  detailed  micro-level 
analysis  is  made  to  call  the  micro-structural  AEH  model 
to  sample  the  local  micro-level  behavior  and  build  the 
element  global  stresses  from  the  constituent  micro- 
stresses. The  details  of  the  coupling  will  be  published  in 
an  ARL  report. 

2.1.  Guidelines  for  using  the  Model. 

The  micro-structural  AEH  option  can  be  specified  by 
using  a Type  3 material  in  PARADYN  input  and  then 
additionally  a non-zero  number  in  the  5Ih  data  on  the  8th 
material  card  of  the  material  data  input.  The  material 
option  describes  a non-linear  elastic-plastic  response  with 
kinematic  and/or  isotropic  hardening.  The  added  micro- 
code is  not  limited  to  micro-phase  materials  of  this  type, 
however. 

Micro-structure  is  next  defined  with  the  usual  nodes, 
elements  and  material  properties.  Although  any  type  of 
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format  can  be  used  for  this  purpose,  the  one  that  is 
selected  uses  the  same  PARADYN  input  format.  This 
was  done  to  avail  PARADYN  pre-processors  such  as 
INGRID.  Although  not  verified  at  present,  there  are  no 
restrictions  on  the  number  of  micro-structural  elements, 
nodes  and  materials. 

All  the  information  defining  the  micro-structural 
AEH  is  to  be  contained  in  an  input  file  separate  from  the 
PARADYN’s  global  structure  defining  input  file  and  is  to 
be  made  available  to  the  PARADYN  executable  along 
with  a domain  decomposition  part  file. 

2.2.  Details  of  the  Data  Exchange. 

While  PARADYN  focuses  on  global  computations 
such  as  the  aggregation  of  nodal  forces,  solution  of  the 
equations  of  motion,  updating  global  nodal  positions, 
enforcing  contact,  etc.,  the  micro-structural  AEH  code 
focuses  on  the  micro-structural  AEH  calculations  which 
typically  involve  evaluating  the  microstmcture  as  a 
perturbation  of  global  deformation  and  periodicity.  For 
the  respective  codes  to  do  their  parts  of  computation,  it  is 
necessary  that  the  relevant  data  be  exchanged  between 
them  for  each  time  step  and  for  each  global  element 
identified  for  micro-computations.  In  the  present  work, 
the  data  exchange  involved  two  parts  as  follows: 

1.  From  PARADYN,  at  the  beginning  of  each  time 
step  and  for  the  identified  global  elements,  the  following 
data  is  written  to  external  processor  specific  data  files  for 
read  access  to  the  micro-structural  AEH  model: 

current  time  step, 
current  global  strain  increments, 
global  strains  from  previous  time  step 
global  stresses  from  previous  time  step 
effective  plastic  strain  from  previous  time  step 

2.  After  reading  the  PARADYN  written  data  files, 
micro-structural  AEH  computes  and  writes  the  following 
data  to  external  processor  specific  data  files  for  read 
access  to  PARADYN: 

Effective  density 
Effective  bulk  modulus 
Effective  shear  modulus 

Computed  current  global  stresses  for  the  element 
Computed  current  effective  plastic  strain  for  the 
element 

While  PARADYN’s  native  arrays  are  used  for 
tracking  global-stresses,  global-strains,  effective 
properties  (density,  bulk  modulus,  shear  strain),  and 
effective  plastic  strain,  no  such  native  arrays  are  used  for 
the  micro-element  quantities.  Instead  these  too  are 
written  to,  and  read  from,  processor  specific  files  during 
each  time  step  and  updated  during  every  time  step. 

As  opposed  to  the  global-stresses  of  the  global 
elements,  the  stored  values  of  these  responses  reflect  the 


accurate  stress  distribution  within  the  micro-elements  of 
the  microstructure.  During  a PARADYN  run,  or  after  the 
run  is  complete,  the  processor  specific  files  can  be 
visualized  independently  for  observing  the  micro- 
structural  stress  and  deformation  development  for  any  of 
the  global  elements. 

3.  Results 

The  following  numerical  analyses  are  conducted  on 
the  ARL’s  SGI  02k  machines  for  verifying  and  validating 
the  model’s  coupling  and  scalability. 

3.1.  Verification  of  the  Model  Coupling. 

j 

For  verifying  the  model  coupling,  a small  cylinder 
with  homogeneous  material  was  considered  in  a Taylor 
impact,  and  was  analyzed  alternatively  as  a homogeneous 
material  with  DYNA3D  and  as  a homogeneous  material 
with  alternating  layers  of  two  different  materials  but  with 
same  identical  properties  using  the  micro-structural  AEH 
in  PARADYN.  The  idea  was  that  the  AEH  model  should 
default  into  a homogeneous  material  solution,  and  any 
discrepancy  that  shows  up  may  point  to  errors  in  the  data 
exchange  in  the  codes’  coupling. 

Enforcing  quarter  symmetry  on  side's  parallel  to  the 
axis  of  the  cylinder,  a 90  degree  wedge!  of  the  cylinder 
was  considered  with  108  elements  in  five  rows  as  shown 
in  Figure  1.  The  length  and  radius  of  the  cylinder  are 
2.592  mm  and  3.2  mm,  respectively.  The  material  was 
considered  elastic-plastic  with  isotropic  hardening.  The 
material  properties  are  as  follows: 


Table  1.  Material  Properties 


Density 

Young’s 

modulus 

Poisson 

Yield 

stress 

Tangent 

modulus 

gm/mmA3 

MPa 

ratio 

MPa 

MPa 

8.90E-03 

117,000 

0.35 

4,000 

1000 

The  velocity  of  the  cylinder  is  227  mm/ms.  To  cause 
the  deformation,  one  end  of  the  cylinder  is  assumed  to  be 
free  of  velocity.  Results  for  axial  velocity,  axial  stress 
and  effective  plastic  strain  are  considered1  at  the  impacted 
end.  | 

They  are  presented  in  Figures  2-4,  respectively  for 
the  homorogeneous  DYNA3D  solution  and  for  the 
homogeneous  micro-structural  AEH-PARADYN 
solution.  Both  solutions  appear  to  be  in  ;agreement  with 
each  other  thus  verifying  the  model  coupling  for  micro- 
structures with  homogeneous  materials,  i 
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3.2.  Validation  of  the  Model  3.3.  Scalability  of  the  Results. 


In  this  example,  a cylinder  similar  to  the  one  in  the 
previous  example  was  considered  but  with  two  materials 
alternating  in  a total  of  96  layers  normal  to  the  cylinder 
axis.  The  length  and  radius  of  the  cylinder  are  31.1  mm 
and  3.2  mm,  respectively.  The  material  was  considered 
elastic-plastic  with  isotropic  hardening.  The  material 
properties  are  as  follows: 


Table  2.  Material  Properties 


Layer 

Density 
kg/m  A 3 

Young’s 

modulus 

GPa 

Poisson 

ratio 

Yield 

stress 

GPa 

Tangent 

modulus 

GPa 

1 

8930 

1.17E+11 

0.35 

4.E+08 

1.E+08 

2 

8930 

9.00E+10 

0.35 

4.E+08 

1.E+08 

Two  solutions  were  obtained.  The  first  solution  was 
a heterogeneous  DYNA3D  solution  obtained  using 
10,368  global  elements.  To  model  the  repeating  48  sets 
of  the  two  dissimilar  materials,  8 rows  of  elements  were 
used  for  each  material  layer.  The  second  solution  was  the 
homogeneous  micro-structural  AEH-PARADYN  solution 
obtained  with  a 1,296  element  global  model.  The 
elements  are  stacked  in  96  layers.  In  contrast  to  the  first 
solution,  a single  layer  of  elements  was  used  for  each 
material.  All  layers  are  assumed  to  be  exhibiting 
homogeneous  composite  behavior  in  the  global  sense  but 
with  micro-structural  AEH  computed  using  a two  layer 
micro-structural  model. 

Enforcing  quarter  symmetry  on  sides  parallel  to  the 
axis  of  the  cylinder,  a 90  degree  wedge  of  the  cylinder 
was  considered.  The  velocity  imposed  on  all  but  one  end 
face  of  the  cylinder  is  227  m/s.  Figures  5 and  6 show  the 
two  models  both  before  and  after  the  impact. 

Solutions  at  a node  near  the  bottom  of  the  cylinder  are 
presented  in  Figures  7-9,  for  axial  velocity,  axial  stress, 
and  effective  plastic  strain. 

For  all  these  responses,  the  micro-structural  AEH 
was  able  to  track  the  DYNA3D’s  heterogeneous 
responses  well.  Deviations  in  the  axial  velocity  and  axial 
stress  can  be  attributed  to  three  factors:  1.  these 
differences  are  near  stress  free  edges  prone  to  be  affected 
by  the  transients;  2.  the  meshes  are  different  in  their 
coarseness  which  affects  the  time  integration  in  the  two 
solutions;  and  3.  the  values  are  node  averaged  but  the 
nodes  are  at  different  locations  in  the  two  models  since 
the  meshes  are  different.  Similarly,  the  differences  in  the 
plastic  strain  predictions  can  be  attributed  to  the  extreme 
length  of  the  time  duration  of  the  analysis  which  makes 
the  differences  to  pile  up. 


As  explained  in  Section  3.2,  due  consideration  is 
given  to  the  fact  that  PARDYN  is  a parallel  code  while 
coupling  the  micro-structural  AEH.  Since  the 
PARADYN  code  has  a well  tested  MPI/OPENMP  based 
computational  logic,  the  coupling  was  done  by  keeping  a 
global  element’s  micro-structural  AEH  computations 
local  to  the  processor  on  which  the  global  element  resides. 
This  means  that  information  such  as  micro-element 
stresses,  strains  and  mechanical  properties  needed  in 
micro-element  computations  are  kept  local  and  not  passed 
into  global  data  structures.  This  means  that  a global 
element’s  micro-stress,  micro-strain  and  micro- 
mechanical properties  are  needed  to  be  written  to,  and 
read  from,  processor-specific  files  in  between  time  steps. 
To  underscore  the  fact  that  the  penalty  associated  with 
this  approach  is  small,  results  from  a limited  scalability 
study  are  presented  in  this  section. 

Taylor  impact  model  of  Section  4.2  is  used  for  this 
purpose.  Two  types  of  scalability  studies,  one  with  a 
fixed  mesh  for  standard  scalability,  and  the  other  with 
varying  mesh  size,  but  with  a fixed  number  of  elements 
per  processor,  for  scaled  scalability  are  performed.  The 
nodes,  elements  and  wall  clock  times  from  these  studies 
are  presented  next. 

In  these  models,  the  increase  in  number  of  elements 
was  achieved  by  adding  element  rows  in  the  cylindrical 
axis  direction  only,  and  not  in  the  hoop  and  radial 
directions. 

The  wall  clock  times  are  plotted  in  Figures  10  and  1 1 . 
The  plot  in  Figure  10  shows  run  times  for  the  122,880 
element  mesh  for  the  standard  scalability  study.  A linear 
scalability  can  be  discerned  from  this  plot. 

On  the  other  hand,  the  plot  in  Figure  1 1 shows  the 
run  times  for  the  scaled  scalability  study.  In  this  study, 
the  mesh  size  was  doubled  with  each  doubling  of  the 
processors.  The  run  times  did  not  remain  constant  as 
were  the  number  of  elements  per  the  processor.  The 
reason  could  be  that  the  plastic  deformation  is  much  more 
concentrated  near  base  and  the  domain  decomposition 
used  to  distribute  the  elements  over  the  processors  is  not 
sensitive  to  this  fact. 
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4.  Future  Work  and  Conclusions 


t 


Micro-structures  are  features  of  the  US  Army’s 
advanced  structures  and  armor  materials. 
Homogenization  theories  such  as  the  AEH  are  being 
developed  to  model  the  effect  of  these  structures  on 
global  material  and  structural  responses.  With  the  present 
coupling  of  the  AEH  based  micro-structural  model  in 
PARADYN,  many  in-situ  material  responses  can  be 
studied  under  diverse  global  structural  conditions.  The 
brief  example  applications  presented  in  the  presentation 
demonstrate  this. 

In  future,  the  following  extensions  to  this  work  will 
be  considered:  1.  using  PARADYN’s  diverse  material 
library  to  model  the  different  micro-phase  materials,  2. 
extending  the  application  range  to  include  complex  micro 
structures  and  diverse  global  applications. 
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Figure  1.  Finite  element  mesh  of  a cylindrical  wedge 
under  Taylor  impact 
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Figure  2.  Axial  velocity  solutions  for  the  finite  element 
mesh  of  Figure  1 j 


Figure  3.  Axial  stress  solutions  for  the  finite  element 
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Figure  5.  Finite  element  model  and  deformed  mesh 
used  for  the  AEH-PARDYN  solution  for  the  verification 
problem 


Figure  6.  Finite  element  model  and  deformed  mesh 
used  for  the  DYNA30  solution  for  the  verification 
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Figure  7.  Axial  velocity  solutions 


Figure  8.  Axial  stress  solutions 


Figure  9.  Effective  plastic  strain  solutions 
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Figure  10.  Linear  Scalability  Study 
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Figure  11.  Scaled  Scalability  Study 
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